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FOREWORD 


Target  identification  using  gray-scale  texture  information  is  of  interest  both  to  the 
surveillance  and  interceptor  design  communities.  This  report  details  an  analog  scheme  to  extract 
texture  features  to  use  for  target  discrimination.  Tlie  features  extracted  from  this  system  could  be 
used  either  as  part  of  an  alert  system  for  a  human  surveillance  team  or  as  part  of  a  terminal  homing 
seeker  in  an  autonomous  kill  vehicle. 

This  work  has  been  supported  by  a  number  of  sponsors  and  has  been  conducted  in  the 
Space  and  Ocean  Geodesy  Branch. 

This  report  has  been  reviewed  by  Dr.  Richard  A.  Lorey,  Head  of  the  Space  and  Ocean 
Geodesy  Branch  and  James  Sloop,  Head  of  the  Space  and  Surface  Systems  Division. 
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ABSTRACT 


This  report  describes  an  analog  method  to  extract  gray-scale  texture  features.  This  method 
uses  a  nonlinear  resistive  grid  to  perform  a  two-dimensional  pseudowavelet  transform  of  the  initial 
input  image.  Texture  features  are  computed  using  first-  and  second-  order  variance  estimates  of  the 
transform  coefficients.  Some  preliminary  results  are  presented  indicating  the  natural  segmentation 
that  the  nonlinearity  provides  at  the  boundary  of  two  dissimilar  textures. 
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INTRODUCTION 


The  silicon  retina  of  Mead*  displays  many  features  of  human  retinal  processing.  The  key 
feature  of  interest  here  is  the  center-surround  response  of  each  node  in  the  retina  that  is  reminiscent 
of  a  mother  wavelet^  This  center-surround  response  is  accomplished  by  differencing  the  input  at  a 
pixel  with  a  local  average  computed  via  an  analog  resistive  grid.  In  this  report,  we  focus  on  the 
utility  of  these  analog  pseudowavelet  coefficients.  We  include  an  example  where  the 
multiresolution  wavelet  coefficients  have  been  further  processed  to  produce  loc^  variance  and 
variance  of  variance  features  that  can  be  used  to  perform  texture  discrimination.  The  full  details  of 
the  hybrid  optoelectronic  implementation  of  this  system  are  addressed  elsewhere.^  The  twin 
advantages  of  this  hybrid  approach  are  the  near  real-time  processing  to  obtain  wavelet  coefficients 
and/or  texture  features  and  the  nonlinear,  data-driven  nature  of  the  effective  wavelets  produced  by  a 
resistive  grid  composed  of  nonlinear  components. 


BACKGROUND 


As  an  outgrowth  of  previous  work  in  texture  discrimination  based  on  fractal  dimension 
derived  power  law  features,"**  5  ^e  have  recently  started  to  consider  using  a  (hypothetical)  silicon 
retina  to  do  image  preprocessing  for  texture  analysis.  There  are  twin  motivations  for  this  line  of 
investigation.  First  and  foremost,  the  texture  discrimination  ability  of  retinal-based  biological 
systems  provides  a  working  prototype  for  this  approach. 

Second,  an  analog  implementation  of  our  hypothetical  retina  offers  the  potential  of  real-time 
preprocessing.  The  postprocessing  is  based  on  well  understood  parametric  and  nonparametric 
statistical  techniques.^  This  permits  an  analysis  of  the  contributions  of  individual  features 
produced  in  the  preprocessing  and  gives  us  a  capability  to  follow  an  evolutionary  or  experimental 
approach  with  our  silicon  retina  model.  We  can  delete  features  that  do  not  perform,  while  trying 
variations  on  new  or  good  features.  Central  to  our  model  is  the  set  of  nonlinearities  encounter^  in 
analog  very  large  scale  integrated  (VLSI)  implementations  as  well  as  in  biological  systems.  These 
nonlinearities  are  crucial,  for  example,  to  both  segmentation  and  the  prevention  of  feature 
contamination  between  disparate  texture  types. 

Our  (hypothetical)  silicon  retina  is  centered  on  the  use  of  two-dimensional  resistive  grids. 
A  resistive  grid  functions  to  smooth  the  input  in  a  linear  or  (of  interest  to  us)  a  nonlinear  fashion. 
In  its  linear  version,  the  resistive  grid  smoothes  the  inputs  to  produce  an  output  that  corresponds 
approximately  to  the  convolution  of  the  input  image  with  an  exponential  kernel  with  a  characteristic 
length  or  kernel  size  we  will  denote  by  Lk  or,  equivalently,  by  an  associated  scale  ek. 
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Let  f:JL2  ->  be  the  function  that  produces  our  two-dimensional  input  image  defined  on  a 
discrete  (pixel)  array.  Further,  let  X  R  ->  R2  x  R,  be  the  resistive  grid  transform  of  f  at 

scale  e.  Suppose  that  we  compute  Re(f)  for  n  different  scales:  R2®(f),  R2Kf).  R2^(f),  R2"‘kf). 

Notationally,  we  let  Lp  =  R2P'*(0-  The  Lp  can  be  thought  of  as  functional  estimates  at  different 
levels  of  smoothing.  Next,  we  compute  a  set  of  features  for  each  pixel: 

Fa  =  R2““kf)  -  R2“(0  =  La  -  La+i  ,  a=l,...,n-l. 

We  now  have  a  set  of  n-1  features  defined  at  each  array  or  grid  (pixel)  point  that 
correspond  to  a  difference  of  kernels  of  different  characteristic  scales.  These  are  the  (nonlinear) 
resistive  grid  analogs  of  the  difference-of-Gaussian  wavelet  basis.  Thus,  at  each  pixel,  we  have 
the  first  n-1  coefficients  of  a  wavelet  (or  pseudowavelet)  basis  with  the  Fa  being  the  coefficient 
arrays. 


Ganglion  cells  in  the  retina  of  the  cat  are  of  two  types,  those  that  respond  to  hyperpolariz- 
ing  signals,  and  those  that  respond  to  depolarizing  signals.^  Each  type  performs  a  half-wave 
rectification  operation  on  the  signal  it  receives.  Carver  Mead  has  demonstrated  the  feasibility  of 
performing  half-  and  full-wave  rectification  in  analog  VLSI.  Therefore,  it  makes  sense  to  produce 
features  based  on  this  physiological  structure  and  analog  VLSI  capability. 

An  obvious  feature  to  compute  using  the  twin  elements  of  resistive  grids  and  full-wave 
rectifiers  is  a  mean  variance  of  the  pseudowavelet  coefficients  as  a  function  of  scale.  The  goal  is  to 
compute  features  that  are  relatively  texture  invariant.  We  compute  these  variance  features  va  as 
follows: 

Va  =  R2a+t  (IFa  -  R2“(Fa)l) 

Here,  x  e  { 1,2,...}  is  chosen  to  smooth  these  variance  estimates-essentially,  this  Iasi  sm<>c/h  '  jv 
analogous  to  basebanding  on  some  carrier  frequency. 

This  gives  a  set  of  additional  features  for  each  array  location  (pixel)  that  give  a  measure  of 
the  variance  of  the  difference  of  kernel  features  for  the  different  scales.  This  set  of  steps  can  be 
repeated  (iterated)  using  vk  as  input  (instead  of  Fk)  to  produce  a  set  of  second  order  variances  vkk- 
Thus,  we  have  a  truncated  double  expansion  in  pseudowavelet  coefficients  and  orders  of  variance 
of  these  coefficients  for  each  array  location  or  pixel. 
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RESULTS 


In  this  section,  a  discussion  of  the  linear  version  of  our  functional  estimation  technique  is 
given  along  with  some  nonlinear  simulation  results.  A  full  analytical  treatment  of  the  nonlinear 
theory  is  beyond  the  scope  of  this  report.  For  analytical  purposes,  consider  a  linear  approximation 
to  our  functional  estimation  technique.  Call  the  exponential  function  ^x)  (see  Figure  1* ),  obtained 
from  the  resistive  grid,  which  is  our  scaling  function.  We  then  have,  as  our  mother  wavelet  v(x), 
the  difference  of  exponentials  depicted  in  Figure  2.  Figure  3  gives  v(x)  at  three  different  scale 
values.  The  effect  of  the  nonlinearity  of  the  (simulated)  analog  VLSI  implementation  on  this 
wavelet  is  depicted  in  Figures  4a  and  4b.  Figure  4a  shows  the  discontinuous  input,  while 
Figure  4b  compares  the  v(x)  function  corresponding  to  the  linear  case  and  the  nonlinear  theory. 
The  effect  of  this  adaptive  wavelet  amounts  to  a  segmentation  effect  in  the  case  where  there  is  a 
sharp  discontinuity  (an  edge)  in  the  input  such  as  in  Figure  4a.  That  these  nonlinear  effects  allow 
for  an  automatic  segmentation  can  be  seen  from  the  ability  of  the  resistive  grid  kernel  estimator 
(RFKE)  to  model  discontinuities  in  probability  density  functions  while  maintaining  an  otherwise 
smooth  estinute  (see  Figures  5  and  6,  reproduced  from  Reference  8).  The  resistive  elements  in  the 
nonlinear  resistive  grid  have  variable  conductance  values.  By  producing  a  conductance  map  of  the 
resistive  grid,  a  natural  segmentation  occurs  as  can  be  seen  in  Figures  7b  and  8b  corresponding  to 
the  input  images  Figures  7a  and  8a,  respectively.  The  dark  areas  in  Figures  7b  and  8b  correspond 
to  drastically  reduced  conductance  values.  These  reduced  conductance  values  serve  to  greatly  limit 
the  contribution  of  one  texture  type  to  the  wavelet  coefficients  centered  in  an  adjacent  texture  type, 
which  is  exactly  the  effect  seen  in  Figure  4b. 

For  the  one-dimensional  example  input  of  Figure  9  (a  vertical  slice  down  the  center,  i.e., 
bushes  to  turret  to  tread  to  grass,  of  the  image  depicted  in  Figure  7a),  this  produces  the  functional 
approximations  Lj(x)  (Figure  9),  and  differencing  Lj  and  Lj-i  will  yield  terms  Fj(x)  analogous  to 
the  wavelet  coefficients  (Figure  10).  Considering  the  deviations  of  Fj(x)  from  a  smoothed 
regression  vj(x)  of  Fj(x)  (Figure  11),  and  the  deviations  of  vj(x)  from  a  smoothed  regression 
vkk(x)  of  vj(x)  (Figure  12),  as  features  indicative  of  texture,  preliminary  discriminant  analysis 
indicates  encouraging  capabilities  to  distinguish  the  different  classes  of  texture. 

Figure  13  depicts  the  high  degree  of  separation  between  the  three  classes  (grass,  bush,  and 
tank)  found  in  Figure  7a  in  a  derived  feature  space  based  on  these  resistive  grid  wavelet 
transformation  featiu^s.  Figures  14  and  15  similarly  indicate  very  good  discriminant  capability  for 
rive  classes  (grass,  trees,  field,  road,  and  building)  found  in  Figure  8a.  Nonparametric 
discriminant  analysis  results  already  show  a  marked  improvement  over  results  that  have  been 
obtained  using  traditional  power  law  features,  and  the  nonlinear  segmentation  effects  appear  to  be 
central  to  these  results.  Feature  analysis  has  shown  that  the  first-  and  second -order  variance 
features  are  of  equal  relative  importance  in  the  texture  discrimination. 

To  illustrate  the  full  power  of  our  approach,  one  more  example  will  be  discussed. 
Figure  16  is  a  255  gray-level  texture  quilt  consisting  of  16  different  textures.  A  similar  image  has 
appeared  previously  in  the  literature  in  conjunction  with  the  unsupervised  texture  discrimination 


*  AH  figures  are  included  after  the  References. 
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work  of  Jain  and  Farrokhnia.^  Each  patch  is  128  x  128  pixels  for  a  total  size  of  512  x  512.  Figure 
17  shows  the  degree  of  separation  obtained  for  the  16  classes  in  a  derived  feature  space.  Each 
class  is  represented  by  250  points  randomly  chosen  in  a  100  x  100  square  positioned  so  its  center 
is  coincident  with  the  center  of  the  patch.  As  can  be  seen  from  Figure  17,  a  high  degree  of 
separability  exists  for  most  of  the  classes.  It  should  also  be  emphasized  that  those  classes 
overlapping  in  this  projection  can  be  separated  in  some  of  the  other  three-dimensional  derived 
feature  spaces. 


CONCLUSION 


Our  approach  seems  to  be  a  powerful  tool  in  identifying  features  useful  in  texture 
repre.sentation  for  (one  type  oO  machine  vision.  In  doing  a  comparison  with  biological  retinas,  we 
are  several  steps  behind  the  ganglia  layer.  Thus,  we  are  in  some  sense  trying  to  find  our  way  from 
the  retinal  preprocessing  backwards  toward  the  higher  processing  levels.  In  finding  what  works  in 
tlic  aiuiK)g  realm,  it  is  hoped  we  will  shed  some  light  on  the  biological  realm  as  well. 

An  electronic  design  for  near  real-time  processing  has  been  presented  to  obtain  wavelet 
coefficients  and/or  texture  features.  Also,  simulation  results  show  that  the  nonlinear,  data-driven 
character  of  the  wavelets  obtained  from  nonlinear  resistive  grids  can  be  an  advantage  rather  than  a 
disadvantage,  especially  where  a  high  contrast  boundary  separates  different  textures.  The 
conversion  of  the  wavelet  coefficient  maps  to  local  power  or  variance  features  has  been 
demonstrated  to  yield  a  useful  set  of  features  for  texture  discrimination. 

Future  work  will  address  the  need  for  comparison  with  more  conventional  wavelet 
techniques,  including  an  analysis  of  the  nonlinear  character  of  the  wavelet  response. 
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RGURE  1.  SCALING  FUNCTION  FOR  ONE-DIMENSIONAL 
RESISTIVE  GRID  WITH  LINEAR  COMPONENTS 


HGURE  2.  MOTHER  WAVELET  v(x)  FOR  ONE-DIMENSIONAL 
RESISTIVE  GRID  WITH  LINEAR  COMPONENTS 
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FIGURE  3.  ONE-DIMENSIONAL  RESISTIVE  GRID  WAVELETS 
AT  THREE  INCREASING  SCALES 


Note:  The  grid  is  allowed  to  come  (o  equilibrium  with  the  step  function  input;  then  resistances 
are  held  Fixed  while  the  spike  is  input.  See  Figure  4b  for  the  effective  resultant  wavelet 
for  the  spike  component  of  the  total  input. 

RGURE  4A.  NONLINEAR  ONE-DIMENSIONAL  RESISTIVE  GRID  INPUT 
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Note:  The  nonlinear  response  strongly  damps  the  wavelet  amplitude  at  a  contrast  boundary. 


HGURE  4B.  EFFECTIVE  RESULTANT  LINEAR  AND  NONLINEAR  WAVELET  FOR 


SPIKE  COMPONENT  OF  TOTAL  INPUT  SHOWN  IN  HGURE  4A 


HGURESA.  ESTIMATE  OF  DENSITY 


BIN  CENTERS 

RGURE  5B.  LINEAR  RGKE  ESTIMATE 


USING  HISTOGRAM 


WITHL=5.0 


FIGURE  5C.  LINEAR  RGKE  ESTIMATE 
WITH  L=2.0 


BIN  CENTERS 

FIGURE  5D.  LINEAR  RGKE  ESTIMATE 
WITHL=I.O 
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BIN  CENTERS 

nGURE6A.  ESTIMATE  OF  DENSITY 
USING  HISTOGRAM 


BIN  CENTERS 

FIGURE  6C.  NONLINEAR  RGKE  ESTIMATE 
WITHp=1.0 


nGURE6B.  NONLINEAR  RGKE  ESTIMATE 
WITH  3=02 


BIN  CENTERS 

nGURE6D.  NONLINEAR  RGKE  ESTIMATE 
WITH  3=2.0 
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FIGURE  7A.  TANK  IMAGE  INPl  TV  TO  NONLINEAR  TWO-DIMENSIONAL  RESISTIVE  GRID 
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Note:  The  conductance  map  has  performed  segmentation  at  all  of  the  higher  contrast 

boundaries.  The  highest  contrast  boundaries  are  typically  due  to  man-made  structures. 

RGURE  8B.  RESULTANT  CONDUCTANCE  MAP  FOR  AERIAL  IMAGE  FOR  MODERATE 

NONLINEARLITY  SETTING 
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Note:  Each  output  has  been  offset  vertically  for  clarity. 


FIGURE  9.  TOP  TO  BOTTOM:  ONE-DIMENSIONAL  INPUT  (TOP  LINE)  PRODUCED 
BY  TAKING  VERTICAL  (TURRET  TO  TREAD)  SLICE  OF  TANK  IMAGE 
OF  FIGURE  7A;  RESISTIVE  GRID  OUTPUTS  AT  SUCCESSIVELY 
DOUBLED  SCALES 
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Note:  Each  output  has  been  offset  vertically  fw  clarity.  The 
input  (top  line)  is  the  same  as  for  Figure  9. 


HGURE  10.  TOP  TO  BOTTOM:  ONE-DIMENSIONAL  INPUT  (TOP  LINE)  PRODUCED 
BY  TAKING  VERTICAL  (TURRET  TO  TREAD)  SLICE  OF  TANK  IMAGE 
OF  RGURE  7A:  SUCCESSIVE  WAVELET  COEEHCIENTS 
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Note:  Each  output  has  been  offset  vertically  for  clarity. 

FIGURE  12.  TOP  TO  BOTTOM;  SECOND-ORDER  ANALOG  REGRESSION 
COEFHCIENTS  FOR  INCREASING  SCALE 
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Legend:  +  Grass 
Bush 


Note:  U1  vs.  U2  (first  two  principal  components) 

1000  observations  per  class 

HGURE  13.  THREE-CLASS  SCATTER  PLOT  OF  DERIVED 
RESISTIVE  GRID  WAVELET  TRANSFORMATION  (RGWT)  DATA 
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Note:  UI  vs.  U2  (Hrst  two  principal  components) 
1000  observations  per  class 


RGURE  14.  BUILDINGS  VS.  ROAD  IN  RGWT  FEATURE  SPACE 
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Legend:  x  Field 

Grass,  Trees,  Building,  Road 


Note:  U1  vs.  U2  (first  two  principal  components) 

1000  observations  per  class 

HGURE  15.  HELDS  VS.  OTHER  IN  RGWT  FEATURE  SPACE 


20 


FIGURE  16.  16 


NSWCDD/TR-92/237 


Note:  Numbering  has  been  done  consecutively  top  to  bottom  across  the  rows. 
nCURE  17.  SEPARATION  OF  16  QUILT  PATCHES  IN  DERIVED  FEATURE  SPACE 
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